Method and apparatus for processing digital service signals

ABSTRACT

An apparatus for processing digital service signals includes a receiving module configured to receive contents having watermarks, where the watermarks are inserted in audio or video frames of the contents, the watermarks carry channel information related to the contents, and content information having URL information for additional data of the contents. An extracting module is configured to extract the watermarks from the audio or video frames, and an obtaining module is configured to obtain the additional data over an IP (Internet Protocol) network using the extracted watermarks.

This application claims the benefit of U.S. Provisional Application No.61/668,437, filed on Jul. 5, 2012, which is hereby incorporated byreference as if fully set forth herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and apparatus for processingan automatic content recognition (ACR) service related to a broadcastprogram, and more particularly, to a method and apparatus for providingan additional service related to broadcast content in an ACRenvironment.

2. Discussion of the Related Art

As digital satellite broadcasting has been introduced, digital databroadcasting has appeared as a new supplementary service. An interactivedata broadcast, which is a representative interactive service, maytransmit not only a data signal but also an existing broadcast signal toa subscriber so as to provide various supplementary services.

A digital data broadcast may be largely divided into an independentservice using a virtual channel and a broadcast-associated service viaan enhanced TV (ETV). The independent service includes only text andgraphics without a broadcast image signal and is provided in a formatsimilar to an existing Internet web page. Representative examples of theindependent service include a weather and stock information provisionservice, a TV banking service, a commercial transaction service, etc.The broadcast-associated service transmits not only a broadcast imagesignal but also additional text and graphic information. A viewer mayobtain information regarding a viewed broadcast program via abroadcast-associated service. For example, there is a service forenabling a viewer to view a previous story or a filming location whileviewing a drama.

In a broadcast-associated service of a digital data broadcast, an ETVservice may be provided based on ACR technology. ACR means technologyfor automatically recognizing content via information hidden in thecontent when a device plays audio/video (A/V content) back.

In implementation of ACR technology, a watermarking or fingerprintingscheme may be used to acquire information regarding content.Watermarking refers to technology for inserting information indicating adigital content provider into digital content. Fingerprinting is equalto watermarking in that specific information is inserted into digitalcontent and is different therefrom in that information regarding acontent purchaser is inserted instead of information regarding a contentprovider.

However, a conventional system for providing an interactive service viaan ACR function has several restrictions.

If a content provider (or a broadcaster) transmits content for areal-time service and enhancement data (or additional data) for an ETVservice and a TV receiver receives the content for the real-time serviceand the enhancement data through external input, the content for thereal-time service may be received but the enhancement data may not bereceived.

In addition, if an existing ETV service is provided on an ACR system,intervention of a separate ACR server (or an ACR solution provider) isinevitably required. Therefore, there is a need for a receiver forperforming an ACR function alone without a separate ACR server.

In addition, in watermarking technology for providing an ETV service, ifthe amount of information included in a watermark is large, quality ofcontent in which a watermark is embedded may be deteriorated. Therefore,there is a limitation in efficient use of watermarking.

In addition, in fingerprinting technology for providing an ETV service,an external ACR server for recognizing content is additionally necessaryand a content recognition result depends on the size of a reference set.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a method and apparatusfor processing an automatic content recognition (ACR) service related toa broadcast program that substantially obviates one or more problems dueto limitations and disadvantages of the related art.

An object of the present invention is to identify information regardinga broadcast program using a broadcast system and to provide additionalinformation of the identified broadcast program in real time.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 is a diagram showing an ACR based ETV service system;

FIG. 2 is a diagram showing an ACR service processing system accordingto an embodiment of the present invention;

FIG. 3 is a diagram showing an ACR service processing system accordingto another embodiment of the present invention;

FIG. 4 is a diagram showing the structure of a receiver according to anembodiment of the present invention;

FIG. 5 is a diagram showing the structure of an ACR service processingsystem according to an embodiment of the present invention;

FIG. 6 is a diagram showing the structure of an ETV service receptionapparatus according to an embodiment of the present invention;

FIG. 7 is a diagram showing the structure of an ETV service receptionapparatus according to another embodiment of the present invention;

FIG. 8 is a diagram showing the structure of an ETV service receptionapparatus according to another embodiment of the present invention;

FIG. 9 is a diagram showing the flow of digital watermarking technologyaccording to an embodiment of the present invention;

FIG. 10 is a diagram showing an ACR query result format according to anembodiment of the present invention;

FIG. 11 is a diagram showing the structure of a content identifier (ID)according to an embodiment of the present invention;

FIG. 12 is a diagram showing operation of a receiver using watermarkingaccording to an embodiment of the present invention;

FIG. 13 is a diagram showing a watermark extracting process according toan embodiment of the present invention;

FIG. 14 is a diagram showing profiling of watermark informationaccording to an embodiment of the present invention;

FIG. 15 is a diagram showing a method of embedding a watermark in avideo frame according to an embodiment of the present invention;

FIG. 16 is a diagram showing a method of embedding a watermark in avideo frame according to another embodiment of the present invention;

FIG. 17 is a diagram showing a method of embedding a watermark in avideo frame according to another embodiment of the present invention;

FIG. 18 is a diagram showing a method of embedding a watermark in anaudio frame according to an embodiment of the present invention;

FIG. 19 is a diagram showing a method of embedding a profiled watermarkaccording to an embodiment of the present invention;

FIG. 20 is a diagram showing a method of embedding watermarks incontinuous frames according to an embodiment of the present invention;

FIG. 21 is a diagram showing a method of extracting watermarks insertedinto continuous frames according to an embodiment of the presentinvention;

FIG. 22 is a diagram showing a method of embedding a watermark in a 3Dvideo frame according to an embodiment of the present invention;

FIG. 23 is a diagram showing the structure of an ACR service provisionsystem according to an embodiment of the present invention;

FIG. 24 is a diagram showing a method of providing an ACR service usinga broadcaster a logo according to an embodiment of the presentinvention;

FIG. 25 is a diagram showing a broadcaster logo according to anembodiment of the present invention;

FIG. 26 is a diagram showing an ACR service using a broadcaster logoaccording to an embodiment of the present invention;

FIG. 27 is a diagram showing an ACR service provision method using alogo and a watermark according to an embodiment of the presentinvention;

FIG. 28 is a diagram showing a method of processing an ACR servicerelated to a broadcast program according to an embodiment of the presentinvention; and

FIG. 29 is a diagram showing a method of processing an ACR servicerelated to a broadcast program according to another embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. The configuration and operation of the embodiments of thepresent invention will be described with reference to the accompanyingdrawings. The configuration and operation of the present invention shownin the drawings and described hereinafter will be described in at leastone embodiment, without limiting the spirit and scope of the presentinvention.

Although the terms used in the present invention are selected fromgenerally known and used terms, terms used herein may be changeddepending on operator's intention or customs in the art, appearance of anew technology, or the like. In addition, some of the terms mentioned inthe description of the present invention have been selected by theapplicant at his or her discretion, the detailed meanings of which aredescribed in relevant parts of the description herein. Furthermore, itis required that the present invention is understood, not simply by theactual terms used but by the meanings of each term lying within.

FIG. 1 is a diagram showing an ACR based ETV service system.

The ACR based ETV service system shown in FIG. 1 may include abroadcaster or content provider 100, a multichannel video programmingdistributor (MVPD) 101, a set-top box (STB) 102, a receiver 103 such asa digital TV receiver, and an ACR server (or an ACR Solution Provider)104. The receiver 103 may operate according to definition of theadvanced television system committee (ATSC) and may support an ACRfunction. A real-time broadcast service 110 may include A/V content.

A digital broadcast service may be largely divided into a terrestrialbroadcast service provided by the broadcaster 100 and a multi-channelbroadcast service, such as a cable broadcast or a satellite broadcast,provided by the MVPD 101. The broadcaster 100 may transmit a real-timebroadcast service 110 and enhancement data (or additional data) 120together. In this case, as shown in FIG. 1, the receiver 103 may receiveonly the real-time broadcast service 110 and may not receive theenhancement data 120 through the MVPD 101 and the STB 102.

Accordingly, in order to receive the enhancement data 120, the receiver103 analyzes and processes A/V content output as the real-time broadcastservice 110 and identifies broadcast program information and/orbroadcast program related metadata. Using the identified broadcastprogram information and/or broadcast program related metadata, thereceiver 103 may receive the enhancement data from the broadcaster 100or the ACR server 104 (140). In this case, the enhancement data may betransmitted via an Internet protocol (IP) network 150.

If the enhancement data is received from a separate ACR server 104(140), in a mechanism between the ACR server 104 and the receiver 103, arequest/response model among triggered declarative object (TDO) modelsdefined in the ATSC 2.0 standard may be applied to the ACR server 104.Hereinafter, the TDO and request/response model will be described.

TDO indicates additional information included in broadcast content. TDOserves to timely triggers additional information within broadcastcontent. For example, if an audition program is broadcast, a currentranking of an audition participant preferred by a viewer may bedisplayed along with the broadcast content. At this time, additionalinformation of the current rating of the audition participant may be aTDO. Such a TDO may be changed through interaction with viewers orprovided according to viewer's intention.

In the request/response ACR model of the standard ATSC 2.0, the digitalbroadcast receiver (103) is expected to generate signatures of thecontent periodically (e.g. every 5 seconds) and send requests containingthe signatures to the ACR server (104). When the ACR server (104) gets arequest from the digital broadcast receiver (103), it returns aresponse. The communications session is not kept open betweenrequest/response instances. In this model, it is not feasible for theACR server (104) to initiate messages to the client.

However, in the above-described ETV service system, an ACR function maynot be provided without a separate ACR server. Therefore, there is aneed for a method of independently performing an ACR function at areceiver without a separate ACR server.

FIG. 2 is a diagram showing an ACR service processing system accordingto an embodiment of the present invention.

According to the ACR service processing system shown in FIG. 2,enhancement data related to a real-time broadcast may be received usingwatermarking without intervention of an ACR server to provide aninteractive service.

The ACR service processing system shown in FIG. 2 may include atransmitter 200 for generating and transmitting a digital service signaland/or a receiver 210 for receiving and processing a digital servicesignal. The transmitter 200 may be a content provider or a broadcasterand the receiver 210 may be a receiver such as a digital TV receiver.The transmitter 200 may include a generating module 201, an insertingmodule 202 and/or a transmitting module 203. The receiver 210 mayinclude a receiving module 211 and/or an extracting module 212.

Hereinafter, operation of each module of the ACR service processingsystem shown in FIG. 2 will be described.

First, the generating module 201 of the transmitter 200 may generate awatermark and insert the watermark into a frame of A/V content to betransmitted. The watermark may include information about a channel ofthe content, into which the watermark is inserted, and/or contentinformation (or metadata) such as URL information of enhancement data ofcontent. The transmitting module 203 may transmit the content, intowhich the watermark is inserted, to the receiver 210.

The receiving module 211 of the receiver 210 may receive the content,into which the watermark is inserted, and deliver the content to theextracting module 212. The extracting module 212 may extract theinserted watermark from an A/V frame of the delivered content.

FIG. 3 is a diagram showing an ACR service processing system accordingto another embodiment of the present invention.

More specifically, FIG. 3 shows an architecture of a broadcast servicesupporting an ACR based ETV service.

As shown in FIG. 3, the ACR service processing system may include acontent provider or broadcaster 300, an MVPD 302, a set-top box 303, areceiver 304, an application URL lookup server (ALS) 306 and anapplication server (AS) 307. The content provider 300 may include awatermark inserter 301 and the receiver 304 may include a watermarkextractor 305. The ALS 306 and the AS 307 are physically subordinate toone server.

The receiver 304 shown in FIG. 3 has an ACR function.

The content provider 300 may insert metadata related to enhancement datainto A/V content transmitted in real time in the form of a watermark.Even when A/V content is received via the MVPD 302, A/V content, intowhich metadata related to enhancement data is inserted in the form of awatermark, may be received. The receiver 304 having the ACR function mayextract the watermark from the A/V content received through the MVPD 302and recognize information about a program broadcast in real time andmetadata. The receiver 304 may receive enhancement data via an IPnetwork.

Hereinafter, operation of each component shown in FIG. 3 will bedescribed.

The broadcaster 304 may insert metadata related to enhancement data suchas a content ID for identifying content into A/V content in the form ofa watermark. More specifically, the watermark inserter 301 searches theA/V content for an area, into which the watermark will be appropriatelyinserted, and inserts the watermark into the searched area.

The receiver 304 having an ACR function may receive A/V content, intowhich the watermark is inserted, through the MVPD 302 and the set-topbox 303, extract the watermark from the A/V content and obtain metadatarelated to the enhancement data. More specifically, the watermarkextractor 305 of the receiver 304 may search the received A/V contentfor the area, into which the watermark is inserted, and extract thewatermark inserted into the searched area.

The receiver 304 may download the enhancement data related to areal-time broadcast program via the metadata obtained from the extractedwatermark. More specifically, the receiver 304 may access the ALS 306via the IP network to receive a URL of an associated applicationaccording to a query. According to the URL of the application, thereceiver 304 may access the AS 306 for managing the application anddownload the enhancement data subordinate to the application from the AS306.

FIG. 4 is a diagram showing the structure of a receiver according to anembodiment of the present invention.

More specifically, FIG. 4 shows an embodiment of the configuration of areceiver supporting an ACR based ETV service using watermarking.

As shown in FIG. 4, the receiver supporting the ACR based ETV serviceaccording to the embodiment of the present invention may include aninput data processor, an ATSC main service processor, an ATSCmobile/handheld (MH) service processor and/or an ACR service processor.The input data processor may include a tuner/demodulator 400 and/or avestigial side band (VSB) decoder 401. The ATSC main service processormay include a transport protocol (TP) demux 402, a Non Real Time (NRT)guide information processor 403, a digital storage media command andControl (DSM-CC) addressable section parser 404, an Information Provider(IP)/User Datagram Protocol (UDP) parser 405, a FLUTE parser 406, ametadata module 407, a file module 408, an electronic service guide(ESG)/data carrier detect (DCD) handler 409, a storage control module410, a file/TP switch 411, a playback control module 412, a first 1storage device 413, an IP packet storage control module 414, an Internetaccess control module 415, an IP interface 416, a live/recorded switch417, a file (object) decoder 418, a TP/Packetized Elementary Stream(PES) decoder 420, a Program Specific Information (PSI)/program andsystem information protocol (PSIP) decoder 421 and/or an ElectronicProgram Guide (EPG) handler 422. The ATSC MH service processor mayinclude a main/MH/NRT switch 419, a MH baseband processor 423, an MHphysical adaptation processor 424, an IP protocol stack 425, a filehandler 426, an ESG handler 427, a second storage device 428 and/or astreaming handler 429. The ACR service processor may include amain/MH/NRT switch 419, an A/V decoder 430, an A/V process module 431,an external input handler 432, a watermark extractor 433 and/or anapplication 434.

Hereinafter, operation of each module of each processor will bedescribed.

In the input data processor, the tuner/demodulator 400 may tune anddemodulate a broadcast signal received from an antenna. Through thisprocess, a VSB symbol may be extracted. The VSB decoder 401 may decodethe VSB symbol extracted by the tuner/demodulator 400.

The VSB decoder 401 may output ATSC main service data and MH servicedata according to decoding. The ATSC main service data may be deliveredto and processed by the ATSC main service processor and the MH servicedata may be delivered to and processed by the ATSC MH service processor.

The ATSC main service processor may process a main service signal inorder to deliver main service data excluding an MH signal to the ACRservice processor. The TP demux 402 may demultiplex transport packets ofATSC main service data transmitted via the VSB signal and deliver thedemultiplexed transport packets to other processing modules. That is,the TP demux 402 may demultiplex a variety of information included inthe transport packets and deliver information such that elements of thebroadcast signal are respectively processed by modules of the broadcastreceiver. The demultiplexed data may include real-time streams, DSM-CCaddressable sections and/or an NRT service table/A/90&92 signalingtable. More specifically, as shown in FIG. 4, the TP demux 402 mayoutput the real-time streams to the live/recorded switch 417, output theDSM-CC addressable sections to the DSM-CC addressable section parser 404and output the NRT service table/A/90&92 signaling table to the NRTguide information processor 403.

The NRT guide information processor 403 may receive the NRT servicetable/A/90&92 signaling table from the TP demux 402 and extract anddeliver FLUT session information to the DSM-CC addressable sectionparser 404. The DSM-CC addressable section parser 404 may receive theDSM-CC addressable sections from the TP demux 402, receive the FLUTsession information from the NRT guide information processor 403 andprocess the DSM-CC addressable sections. The IP/UDP parser 405 mayreceive the data output from the DSM-CC addressable section parser 404and parse IP datagrams transmitted according to the IP/UDP. The FLUTEparser 406 may receive data output from the IP/UDP parser 405 andprocess FLUTE data for transmitting a data service transmitted in theform of an asynchronous layered coding (ALC) object. The metadata module407 and the file module 408 may receive the data output from the FLUTEparser 406 and process metadata and a restored file. The ESG/DCD handler409 may receive data output from the metadata module 407 and process anelectronic service guide and/or downlink channel descriptor related to abroadcast program. The restored file may be delivered to the storagecontrol module 410 in the form of a file object such as ATSC 2.0 contentand reference fingerprint. The file object may be processed by thestorage control module 410 and divided into a normal file and a TP fileto be stored in the first storage device 413. The playback controlmodule 412 may update the stored file object and deliver the file objectto the file/TP switch 411 in order to decode the normal file and the TPfile. The file/TP switch 411 may deliver the normal file to the filedecoder 418 and deliver the TP file to the live/recorded switch 417 suchthat the normal file and the TP file are decoded through differentpaths.

The file decoder 418 may decode the normal file and deliver the decodedfile to the ACR service processor. The decoded normal file may bedelivered to the main/MH/NRT switch 419 of the ACR service processor.The TP file may be delivered to the TP/PES decoder 420 under the controlof the live/recorded switch 417. The TP/PES decoder 420 decodes the TPfile and the PSI/PSIP decoder 421 decodes the decoded TP file again. TheEPG handler 422 may process the decoded TP file and process an EPGservice according to ATSC.

The ATSC MH service processor may process the MH signal in order totransmit ATSC MH service data to the ACR service processor. Morespecifically, the MH baseband processor 423 may convert the ATSC MHservice data signal into a pulse waveform suitable for transmission. TheMH physical adaptation processor 424 may process the ATSC MH servicedata in a form suitable for an MH physical layer.

The IP protocol stack module 425 may receive the data output from the MHphysical adaption processor 424 and process data according to acommunication protocol for Internet transmission/reception. The filehandler 426 may receive the data output from the IP protocol stackmodule 425 and process a file of an application layer. The ESG handler427 may receive the data output from the file handler 426 and process amobile ESG. In addition, the second storage device 428 may receive thedata output from the file handler 426 and store a file object. Inaddition, some of the data output from the IP protocol stack module 425may become data for an ACR service of the receiver instead of a mobileESG service according to ATSC. In this case, the streaming handler 429may process real streaming received via a real-time transport protocol(RTP) and deliver the real streaming to the ACR service processor.

The main/MH/NRT switch 419 of the ACR service processor may receive thesignal output from the ATSC main service processor and/or the ATSC MHservice processor. The A/V decoder 430 may decode compression A/V datareceived from the main/MH/NRT switch 419. The decoded A/V data may bedelivered to the A/V process module 431.

The external input handler 432 may process the A/V content receivedthrough external input and transmit the A/V content to the A/V processmodule 431.

The A/V process module 431 may process the A/V data received from theA/V decoder 430 and/or the external input handler 432 to be displayed ona screen. In this case, the watermark extractor 433 may extract datainserted in the form of a watermark from the A/V data. The extractedwatermark data may be delivered to the application 434. The application434 may provide an enhancement service based on an ACR function,identify broadcast content and provide enhancement data associatedtherewith. If the application 434 delivers the enhancement data to theA/V process module 431, the A/V process module 431 may process thereceived A/V data to be displayed on a screen.

FIG. 5 is a diagram showing the structure of an ACR service processingsystem according to an embodiment of the present invention.

More specifically, FIG. 5 shows operation for acquiring programinformation using a watermark if a receiver 500 receives a terrestrialchannel through external input of an external device such as a set-topbox.

The ACR service processing system shown in FIG. 5 may include a receiver500, a CE server 501, a broadcaster or content provider 502 and anapplication server 503. The receiver 500 may include an A/V processor510, a watermark extractor 520 and an application module 530. The CEserver 501 may include a watermarking manager 511. In addition, thereceiver 500 may have an ACR function.

The A/V processor 510 delivers A/V content received through externalinput to the watermark extractor 520 and the watermark extractor 520 mayextract data inserted in the form of a watermark from the A/V contentand access the application server 503. At this time, the CE server 501may manage a watermarking related module and micro-code.

The micro-code according to one embodiment of the present inventionrefers to code for extracting a watermark inserted into A/V content andmay be dynamically downloaded by an A/V decoder chip and/or a raw dataprocessing chip. The receiver may compress and store one or a pluralityof micro-codes as necessary or download one or a plurality ofmicro-codes through a download interface. The download interface may bedifferently implemented according to chip type.

Hereinafter, operation of each component shown in FIG. 5 will bedescribed.

The watermark extractor 520 of the receiver 500 may extract a watermarkfrom audio content or video content or extract a watermark using audiocontent and video content. The extracted watermark may include channelinformation and/or content information. The content information mayinclude information about content used in an ATSC 2.0 service.

The application module 530 may tune to a channel for transmittingcontent associated with the channel and/or metadata based on channelinformation and content information extracted from the watermark andreceive content information. In addition, the application module 530 mayreceive content and/or metadata associated with the channel via an IPnetwork. According to one embodiment of the present invention, thechannel may include an ATSC message handler (MH) channel.

The CE server 501 may manage and update a watermarking relatedmicro-code and/or software. That is, the CE server 501 may continuouslyadd, delete or update related micro-code and/or software according to aprotocol of the broadcaster 500. If a request is received from thereceiver 500 or the CE server 501 is updated, the CE server 501 mayinform the receiver 500 that the request is received or the CE server501 is updated and download a new code.

The watermark extractor 520 may be implemented as micro-code or softwaremodule operating on a chip of the receiver 500. The new micro-code orsoftware module may be continuously downloaded from the CE server 501.

FIGS. 6 to 8 are diagrams showing the structure of an ETV servicereception apparatus according to an embodiment of the present invention.

FIGS. 6 to 8 are diagrams showing the structure of a receiver forextracting a watennark inserted into content in real time based onmicro-code in a chip of a video decoder and/or an audio decoder andrecognizing content based on the watermark. Hereinafter, the embodimentof the present invention will be described with reference to FIGS. 6 to8.

FIG. 6 is a diagram showing the structure of an ETV service receptionapparatus according to an embodiment of the present invention.

The ETV service reception apparatus according to the embodiment of thepresent invention shown in FIG. 6 may include a remote controller 600, aremote controller receiver 601, a UI module 602, a network interface603, a micro-code downloader 604, a web browser 605, a trigger processor606, a micro processor 607, a micro-code interface (mit) 608, an audiodecoder 609, a video decoder 610, a graphics processor 611, anapplication 612, a speaker 613, a display module 614, an antenna 615, atuner 616, a demodulator 611, and/or a system decoder 618. The systemdecoder 618 may be an MPEG-2 TS decoder. The demodulator 617 may be aVSB demodulator or a digital video broadcasting (DVB) demodulator. Eachof the audio decoder 609 and the video decoder 610 may include amicro-code interface 608.

As shown in FIG. 6, the ETV service reception apparatus may extract awatermark from A/V content transmitted through a terrestrial wave inreal time using micro-code running on a chip of a video decoder or anaudio decoder, transmit the extracted watermark to an application andreceive an enhancement service. The micro-code may be downloaded from anexternal device through the micro-code downloader 604. The micro-codemay be dynamically downloaded on the chip of the video decoder 609 orthe audio decoder 610 via the micro-code interface 608.

Hereinafter, operation between components of the ETV service receptionapparatus shown in FIG. 6 will be described.

The remote controller receiver 601 may receive a request signal of aviewer through infrared (IR) light from the remote controller 600 andtransmit the signal to the UI module 602 using a key code. The UI module602 and/or the remote controller receiver 601 may deliver the signal tothe web browser 605 using a browser key code. The signal passing throughthe UI module 602 may be delivered to the graphics processor 601.

The tuner 616 tunes the signal received from the antenna 615 and thedemodulator 617 may restore the waveform of the tuned signal. Thedecoder 618 may decode the restored signal and deliver the decodedsignal to the trigger processor 606. The trigger processor 606 mayprocess the decoded signal to trigger data and transmit the trigger datato the web browser 605.

If the signal delivered to the web browser 605 is delivered to themicro-code downloader 604, the micro-code may be downloaded anddelivered to the micro-code interface 608. The micro-code passingthrough the micro-code interface 608 and the decoded signal may bedivided into an audio signal and a video signal which are respectivelydelivered to the audio decoder 609 and the video decoder 610. Themicro-code interface 619 of the audio decoder 609 and the video decoder610 may extract the watermark inserted into A/V content using themicro-code.

The extracted watermark may be delivered to the application 612, theaudio signal may be delivered to the speaker 613 and the video signalmay be output to the display module 614 along with the graphic signalreceived from the graphics processor 611.

FIG. 7 is a diagram showing the structure of an ETV service receptionapparatus according to another embodiment of the present invention.

The ETV service reception apparatus according to the embodiment of thepresent invention shown in FIG. 7 may include a remote controller 700, aremote controller receiver 701, a UI module 702, a network interface703, a micro-code downloader 704, a web browser 705, a trigger processor706, a micro processor 707, a micro-code interface (mif) 708, an audiodecoder 709, a video decoder 710, a graphics processor 711, anapplication 712, a speaker 713, a display module 714, an antenna 715, atuner 716, a demodulator 717, a system decoder 718, an audiopostprocessor and amplifier 719 and/or a video display processor 720.According to the embodiment of the present invention, the system decoder718 may include an MPEG-2 TS decoder. According to the embodiment of thepresent invention, the demodulator 717 may include a VSB demodulatorand/or a DVB demodulator. Each of the audio postprocessor and amplifier719 and the video display processor 720 may include the micro-codeinterface 708.

As shown in FIG. 7, the ETV service reception apparatus may extract awatermark inserted into A/V content through micro-code in real time on apost-processing chip instead of the chip of the audio decoder or thevideo decoder. According to the embodiment of the present invention, thepostprocessing chip may include the audio postprocessor and amplifier719 and the video display processor 720. The ETV service receptionapparatus may automatically recognize the A/V content using theextracted watermark. The ETV service reception apparatus may downloadthe micro-code from an external device through the micro-code downloader704. In addition, the audio or video postprocessing chip of the ETVservice reception apparatus may download the micro-code through themicro-code interface 708.

Hereinafter, operation between the components of the ETV servicereception apparatus shown in FIG. 7 will be described.

The remote controller receiver 701 may receive a request signal of aviewer through infrared (IR) light from the remote controller 700 andtransmit the received signal to the UI module 702 using a key code. TheUI module 702 and/or the remote controller receiver 701 may deliver thesignal to the web browser 705 using a browser key code. The signalpassing through the UT module 702 may be delivered to the graphicsprocessor 701.

The tuner 716 tunes the signal received from the antenna 715 and thedemodulator 717 may restore the waveform of the tuned signal. Thedecoder 718 may decode the restored signal and deliver the decodedsignal to the trigger processor 706. The trigger processor 706 mayprocess the decoded signal into trigger data and transmit the triggerdata to the web browser 705.

If the signal delivered to the web browser 705 is delivered to themicro-code downloader 704, the micro-code may be downloaded anddelivered to the micro-code interface 708 of the audio postprocessor andamplifier 718 and/or the display processor 720 through the micro-codeinterface 708.

The decoded signal may be divided into an audio signal and a videosignal, which are respectively delivered to the audio decoder 709 andthe video decoder 710. The audio signal decoded by the audio decoder 709may be delivered to the audio postprocessor and amplifier 719. The videosignal decoded by the video decoder 710 may be delivered to the videodisplay processor 720.

If the micro-code interface 708 of the audio postprocessor and amplifier718 and/or the display processor 720 receives the audio signal and/orthe video signal through the micro-code interface 708, the watermarkinserted into audio and/or video content may be extracted through themicro-code.

The extracted watermark may be delivered to the application 712, theaudio signal may be output to the speaker 713 and the video signal maybe output to the display module 714.

FIG. 8 is a diagram showing the structure of an ETV service receptionapparatus according to another embodiment of the present invention.

The ETV service reception apparatus according to the embodiment of thepresent invention shown in FIG. 8 may include a remote controller 800, aremote controller receiver 801, a UI module 802, a network interface803, a micro-code downloader 804, a web browser 805, a trigger processor806, a micro processor 807, a micro-code interface (mif) 808, an audiodecoder 809, a video decoder 810, a graphics processor 811, anapplication 812, a speaker 813, a display module 814, an audiopostprocessor and amplifier 815, a set-top box 817 and an external inputhandler 818. The external input handler 818 may receive external inputthrough a high definition multimedia interface (HDMI).

As shown in FIG. 8, the ETV service reception apparatus mayautomatically recognize A/V content using micro-code and receive an ETVservice even when the A/V content is not transmitted through aterrestrial wave but is received via an external device such as theset-top box 817. That is, if A/V content is received from an externaldevice, the ETV service reception apparatus may extract a watermark fromthe A/V content using micro-code in real time. At this time, a video oraudio post-processing chip may download the micro-code through themicro-code interface 808. The extracted watermark may be delivered tothe application 812 and the receiver may acquire enhancement data usingdata included in the watermark. According to the embodiment of thepresent invention, the post-processing chip may include the audiopostprocessor and amplifier 815 and/or the video display processor 816.

Hereinafter, operation between the components of the ETV servicereception apparatus shown in FIG. 8 will be described.

The remote controller receiver 801 may receive a request signal of aviewer through infrared (IR) light from the remote controller 800 andtransmit the received signal to the UI module 802 using a key code. TheUI module 802 and/or the remote controller receiver 801 may deliver thesignal to the web browser 805 using a browser key code. If the signaldelivered to the web browser 805 is delivered to the micro-codedownloader 804, the ETV service reception apparatus may download themicro-code and deliver the micro-code to the micro-code interface 808.The signal passing through the UI module 802 may be delivered to thegraphics processor 8701.

The signal received from the external device through the set-top box maybe processed into trigger data through the trigger processor 806 withoutbeing decoded through the external input handler 818 and may bedelivered to the web browser 805. Alternatively, the external inputsignal passing through the external input handler 818 may be dividedinto an audio signal and a video signal which are respectively deliveredto the audio decoder 809 and the video decoder 810. At this time, theaudio decoder 809 may decode the audio signal.

The decoded audio signal may be delivered to the audio postprocessor andamplifier 819 and the decoded video signal may be delivered to the videodisplay processor 816. The micro-code interface 808 of the audiopostprocessor and amplifier 819 and the display processor 816 mayextract the watermark inserted into audio and/or video content using themicro-code.

The extracted watermark may be delivered to the application 812, theaudio signal may be output to the speaker 813 and the video signal maybe output to the display module 814.

FIG. 9 is a diagram showing the flow of digital watermarking technologyaccording to an embodiment of the present invention.

Digital watermarking is the process of embedding information into adigital signal in a way that is difficult to remove. The signal may beaudio, pictures or video, for example. If the signal is copied, then theinformation is also carried in the copy. A signal may carry severaldifferent watermarks at the same time.

In visible watermarking, the information is visible in the picture orvideo. Typically, the information is text or a logo which identifies theowner of the media. When a television broadcaster adds its logo to thecorner of transmitted video, this is also a visible watermark.

In invisible watermarking, information is added as digital data toaudio, picture or video, but it cannot be perceived as such, although itmay be possible to detect that some amount of information is hidden. Thewatermark may be intended for widespread use and is thus made easy toretrieve or it may be a form of Steganography, where a partycommunicates a secret message embedded in the digital signal. In eithercase, as in visible watermarking, the objective is to attach ownershipor other descriptive information to the signal in a way that isdifficult to remove. It is also possible to use hidden embeddedinformation as a means of covert communication between individuals.

One application of watermarking is in copyright protection systems,which are intended to prevent or deter unauthorized copying of digitalmedia. In this use a copy device retrieves the watermark from the signalbefore making a copy; the device makes a decision to copy or notdepending on the contents of the watermark. Another application is insource tracing. A watermark is embedded into a digital signal at eachpoint of distribution. If a copy of the work is found later, then thewatermark can be retrieved from the copy and the source of thedistribution is known. This technique has been reportedly used to detectthe source of illegally copied movies.

Annotation of digital photographs with descriptive information isanother application of invisible watermarking.

While some file formats for digital media can contain additionalinformation called metadata, digital watermarking is distinct in thatthe data is carried in the signal itself.

The information to be embedded is called a digital watermark, althoughin some contexts the phrase digital watermark means the differencebetween the watermarked signal and the cover signal. The signal wherethe watermark is to be embedded is called the host signal.

A watermarking system is usually divided into three distinct steps,embedding 901, attack 902 and detection (often called extraction) 903.

In embedding 901, an algorithm accepts the host and the data to beembedded and produces a watermarked signal.

The watermarked signal is then transmitted or stored, usuallytransmitted to another person. If this person makes a modification, thisis called an attack 902. While the modification may not be malicious,the term attack arises from copyright protection application, wherepirates attempt to remove the digital watermark through modification.There are many possible modifications, for example, lossy compression ofthe data, cropping an image or video, or intentionally adding noise.

Detection 903 is an algorithm which is applied to the attacked signal toattempt to extract the watermark from it. If the signal was unmodifiedduring transmission, then the watermark is still present and it can beextracted. In robust watermarking applications, the extraction algorithmshould be able to correctly produce the watermark, even if themodifications were strong. In fragile watermarking, the extractionalgorithm should fail if any change is made to the signal.

A digital watermark is called robust with respect to transformations ifthe embedded information can reliably be detected from the marked signaleven if degraded by any number of transformations. Typical imagedegradations are JPEG compression, rotation, cropping, additive noiseand quantization. For video content temporal modifications and MPEGcompression are often added to this list. A watermark is calledimperceptible if the watermarked content is perceptually equivalent tothe original, unwatermarked content. In general it is easy to createrobust watermarks or imperceptible watermarks, but the creation ofrobust and imperceptible watermarks has proven to be quite challenging.Robust imperceptible watermarks have been proposed as tool for theprotection of digital content, for example as an embedded‘no-copy-allowed’ flag in professional video content.

Digital watermarking techniques can be classified in several ways.

First, a watermark is called fragile if it fails to be detected afterthe slightest modification (Robustness). Fragile watermarks are commonlyused for tamper detection (integrity proof). Modifications to anoriginal work that are clearly noticeable are commonly not referred toas watermarks, but as generalized barcodes. A watermark is calledsemi-fragile if it resists benign transformations but fails detectionafter malignant transformations. Semi-fragile watermarks are commonlyused to detect malignant transformations. A watermark is called robustif it resists a designated class of transformations. Robust watermarksmay be used in copy protection applications to carry copy and accesscontrol information.

Second, a watermark is called imperceptible if the original cover signaland the marked signal are (close to) perceptually indistinguishable(Perceptibility). A watermark is called perceptible if its presence inthe marked signal is noticeable, but non-intrusive.

Third, about a capacity, the length of the embedded message determinestwo different main classes of watermarking schemes:

The message is conceptually zero-bit long and the system is designed inorder to detect the presence or the absence of the watermark in themarked object. This kind of watermarking schemes is usually referred toas Italic zero-bit or Italic presence watermarking schemes. Sometimes,this type of watermarking scheme is called 1-bit watermark, because a 1denotes the presence (and a 0 the absence) of a watermark.

The message is a n-bit-long stream (, with n=|m|) or M={0,1}n and ismodulated in the watermark. This kinds of schemes are usually referredto as multiple bit watermarking or non zero-bit watermarking schemes.

Forth, there are several ways for the embedding step. A watermarkingmethod is referred to as spread-spectrum if the marked signal isobtained by an additive modification. Spread-spectrum watermarks areknown to be modestly robust, but also to have a low information capacitydue to host interference. A watermarking method is said to be ofquantization type if the marked signal is obtained by quantization.Quantization watermarks suffer from low robustness, but have a highinformation capacity due to rejection of host interference.

A watermarking method is referred to as amplitude modulation if themarked signal is embedded by additive modification which is similar tospread spectrum method but is particularly embedded in the spatialdomain.

FIG. 10 is a diagram showing an ACR query result format according to anembodiment of the present invention.

According to the existing ACR service processing system, if abroadcaster transmits content for a real-time service and enhancementdata for an ETV service together and a TV receiver receives the contentand the ETV service, the content for the real-time service may bereceived but the enhancement data may not be received.

In this case, according to the embodiment of the present invention, itis possible to solve problems of the existing ACR processing systemthrough an independent IP signaling channel using an IP network. Thatis, a TV receiver may receive content for a real-time service via anMVPD and receive enhancement data via an independent IP signalingchannel.

In this case, according to the embodiment of the present invention, anIP signaling channel may be configured such that a PSIP stream isdelivered and processed in the form of a binary stream. At this time,the IP signaling channel may be configured to use a pull method or apush method.

The IP signaling channel of the pull method may be configured accordingto an HTTP request/response method. According to the HTTPrequest/response method, a PSIP binary stream may be included in an HTTPresponse signal for an HTTP request signal and transmitted throughSignalingChannelURL. In this case, a polling cycle may be periodicallyrequested according to Polling_cycle in metadata delivered as an ACRquery result. In addition, information about a time and/or a cycle to beupdated may be included in a signaling channel and transmitted. In thiscase, the receiver may request signaling information from a server basedon update time and/or cycle information received from the IP signalingchannel.

The IP signaling channel of the push method may be configured using anXMLHTTPRequest application programming interface (API). If theXMLHTTPRequest API is used, it is possible to asynchronously receiveupdates from the server. This is a method of, at a receiver,asynchronously requesting signaling information from a server through anXMLHTTPRequest object and, at the server, providing signalinginformation via this channel in response thereto if signalinginformation has been changed. If there is a limitation in standby timeof a session, a session timeout response may be generated and thereceiver may recognize the session timeout response, request signalinginformation again and maintain a signaling channel between the receiverand the server.

In order to receive enhancement data through an IP signaling channel,the receiver may operate using watermarking and fingerprinting.Fingerprinting refers to technology for inserting information about acontent purchaser into content instead of a content provider. Iffingerprinting is used, the receiver may search a reference database toidentify content. A result of identifying the content is called an ACRquery result. The ACR query result may include a query provided to a TVviewer and answer information of the query in order to implement an ACRfunction. The receiver may provide an ETV service based on the ACR queryresult.

Information about the ACR query result may be inserted/embedded into/inA/V content on a watermark based ACR system and may be transmitted. Thereceiver may extract and acquire ACR query result information through awatermark extractor and then provide an ETV service. In this case, anETV service may be provided without a separate ACR server and a querythrough an IP network may be omitted.

FIG. 10 is a diagram of an XML scheme indicating an ACR query resultaccording to an embodiment of the present invention. As shown in FIG.10, the XML format of the ACR query result may include a result codeelement 1010 and the ACR query result type 1000 may include a content IDelement 1001, a network time protocol (NTP) timestamp element 1002, asignaling channel information element 1003, a service informationelement 1004 and an other-identifier element 1005. The signaling channelinformation element 1003 may include a signaling channel URL element1013, an update mode element 1023 and a polling cycle element 1033, andthe service information element 1004 may include a service name element1014, a service logo element 1024 and a service description element1034.

Hereinafter, the diagram of the XML schema of the ACR query result shownin FIG. 10 will be described in detail and an example of the XML schemawill be described.

The result code element 1010 may indicate a result value of an ACRquery. This may indicate query success or failure and a failure reasonif a query fails in the form of a code value. For example, if the valueof the result code element 1010 is 200, this may indicate that a querysucceeds and content information corresponding thereto is returned and,if the value of the result code element 1010 is 404, this may indicatethat content is not found.

The content ID element 1001 may indicate an identifier for globally anduniquely identifying content and may include a global service identifierelement, which is an identifier for identifying a service.

The NTP timestamp element 1002 may indicate that a time of a specificpoint of a sample frame interval used for an ACR query is provided inthe form of an NTP timestamp. Here, the specific point may be a startpoint or end point of the sample frame. NTP means a protocol forsynchronizing a time of a computer with a reference clock through theInternet and may be used for time synchronization between a time serverand client distributed on a computer network. Since NTP uses a universaltime coordinated (UTC) time and ensures accuracy of 10 ms, the receivermay accurately process a frame synchronization operation.

The signaling channel information element 1003 may indicate accessinformation of an independent signaling channel on an IP network for anETV service.

More specifically, the signaling channel URL element 1013, which is asub element of the signaling channel information element 1003, mayindicate URL information of a signaling channel. The signaling channelURL element 1013 may include an update mode element 1023 and a pollingcycle element 1033 as sub elements. The update mode element 1023 mayindicate a method of acquiring information via an IP signaling channel.For example, in a pull mode, the receiver may periodically performpolling according to a pull method to acquire information and, in a pushmode, the server may transmit information to the receiver according to apush method. The polling cycle element 1033 may indicate a basic pollingcycle value of the receiver according to a pull method if the updatemode element 1023 is a pull mode. Then, the receiver may specify a basicpolling cycle value and transmit a request signal to the server at arandom time interval, thereby preventing requests from overloading inthe server.

The service information element 1004 may indicate information about abroadcast channel. The content ID element 1001 may indicate anidentifier of a service which is currently being viewed by a viewer andthe service information element 1004 may indicate detailed informationabout the broadcast channel. For example, the detailed informationindicated by the service information element 1004 may be a channel name,a logo, or a text description.

More specifically, the service name element 1014 which is a sub elementof the service information element 1004 may indicate a channel name, theservice logo element 1024 may indicate a channel logo, and the servicedescription element 1034 may indicate a channel text description.

The following shows the XML schema of elements of the ACR query resultshown in FIG. 10 according to the embodiment of the present invention.

<xs:complexType name=“ACR-ResultType”>   <xs:sequence>    <xs:elementname=“ContentID” type=“xs:anyURI”/>    <xs:element name=“NTPTimestamp”type=“xs:unsignedLong”/>    <xs:elementname=“SignalingChannelInformation”>     <xs:complexType>     <xs:sequence>       <xs:element name=“SignalingChannelURL”maxOccurs=“unbounded”>        <xs:complexType>        <xs:simpleContent>          <xs:extension base=“xs:anyURI”>          <xs:attribute name=“UpdateMode”>            <xs:simplcTypc>            <xs:restriction base=“xs:string”>             <xs:enumeration value=“Pull”/>              <xs:cnumcrationvaluc=“Push”/>             </xs:restriction>            </xs:simpleType>          </xs:attribute>           <xs:attribute name=“PollingCycle”type=“xs:unsignedInt”/>          </xs:extension>        </xs:simpleContent>        </xs:complexType>       </xs:element>     </xs:sequence>     </xs:complexType>    </xs:element>   <xs:element name=“ServiceInformation”>     <xs:complexType>     <xs:sequence>       <xs:element name=“ServiceName”type=“xs:string”/>       <xs:element name=“ServiceLogo” type=“xs:anyURI”minOccurs=“0”/>       <xs:element name=“ServiceDescription”type=“xs:string” minOccurs=“0” maxOccurs=“unbounded”/>     </xs:sequence>     </xs:complexType>    </xs:element>    <xs:anynamespace=“##other” processContents=“skip” minOccurs=“0”maxOccurs=“unbounded”/>   </xs:sequence>   <xs:attributename=“ResultCode” type=“xs:string” use=“required”/>   <xs:anyAttributeprocessContents=“skip”/>  </xs:complexType>

FIG. 11 is a diagram showing the structure of a content identifier (ID)according to an embodiment of the present invention.

FIG. 11 shows the syntax structure of the content ID according to theATSC standard according to the embodiment of the present invention. TheATSC content ID may be used as an identifier for identifying contentreceived by the receiver.

The ATSC Content Identifier is a structure that is composed of a TSID(Transmitting Subscriber Identification) and a “house number” with aperiod of uniqueness. A “house number” is any number that the holder ofthe TSID wishes as constrained herein. Numbers are unique for each valueof TSID. The syntax of the ATSC Content Identifier structure shall be asdefined in FIG. 11.

‘TSID’, a 16 bit unsigned integer field, shall contain a value oftransport_stream_id. The assigning authority for these values for theUnited States is the FC Ranges for Mexico, Canada, and the United Stateshave been established by formal agreement among these countries. Valuesin other regions are established by appropriate authorities.

‘end_of_day’ field, this 5-bit unsigned integer shall be set to the hourof the day in UTC in which the broadcast day ends and the instant afterwhich the content_id values may be re-used according to unique_for. Thevalue of this field shall be in the range of 0-23. The values 24-31 areserved. Note that the value of this field is expected to be static perbroadcaster.

‘unique_for’ field, this 9-bit unsigned integer shall be set to thenumber of days, rounded up, measure relative to the hour indicated byend_of_day, during which the content_id value is not reassign todifferent content. The value shall be in the range 1 to 511. The valuezero shall be forbidden. The value 511 shall have the special meaning of“indefinitely”. Note that the value of this field is expected to beessentially static per broadcaster, only changing when the method ofhouse numbering is changed. Note also that decoders can treat storedcontent_values as unique until the unique_for fields expire, which canbe implemented by decrementing all stored unique_for fields by one everyday at the end_of_day until they reach zero.

‘content_id’ field, this variable length field shall be set to the valueof the identifier according to the house number system or systems forthe value of TSID. Each such value shall not be assigned to differentcontent within the period of uniqueness set by the values in theend_of_day an unique_for fields. The identifier may be any combinationof human readable and/or binary values and need not exactly match theform of a house number, not to exceed 242 bytes 1.

FIG. 12 is a diagram showing operation of a receiver using watermarkingaccording to an embodiment of the present invention.

As a method of providing an ETV service supporting an ACR function, inthe present invention, a method using watermarking is proposed. FIG. 12is a diagram showing operation between modules of a receiver forperforming an ACR function using watermarking.

As shown in FIG. 12, a receiver using watermarking may include aMobile/Handheld (M/H) receiving module 1200, an Internet module 1201, astoring module 1202, an external input module 1203, a watermarkextracting module 1204, an application module 1205, an A/V processingmodule 1206 and a displaying module 1207.

As shown in FIG. 12, the receiver may recognize a content provider (CP)using a watermark and process an enhancement service provided by therecognized CP.

In the receiver according to the embodiment of the present invention,the watermark extracting module 1204 may extract a watermark from an A/Vsignal received by the M/H receiving module 1200 through external inputand obtain content information included in the A/V signal. The contentinformation, which may be included in the watermark according to theembodiment of the present invention, will be described below.

The extracted content information may be displayed to a viewer throughthe application module 1205 and the viewer may selectively receiveenhancement data for an enhancement service through M/H or Internetusing the displayed content information.

Although not shown in FIG. 12, the received enhancement data may bestored in the storing module 1202 to be used for an ACR service. In thiscase, the content information and metadata are stored in the storingmodule 1202 and the stored content information and metadata may be usedto receive related enhancement data upon A/V playback. Alternatively,the receiver may access the content provider or newly receiveenhancement data through M/H tuning upon A/V playback.

Hereinafter, operation of each module shown in FIG. 12 will be describedand then content information which may be extracted from the watermarkwill be described.

The M/H receiving module 1200 may receive enhancement data from thecontent provider through an M/H signal.

The Internet module 1201 may receive enhancement data from the contentprovider via the Internet.

The storing module 1201 may store data including A/V information and/orenhancement data received from the content provider. If the watermarkextracting module 1204 extracts a watermark, the watermark may bedisplayed and stored in the storage device as special metadata.

The external input module 1203 may receive an A/V signal of the contentprovider through external input.

The watermark extracting module 1204 may extract a watermark from theA/V signal received by the external input module 1203. If content isstored in the storage of the TV and is then played back, the watermarkmay be extracted upon playback.

The application module 1205 may receive the watermark extracted by thewatermark extracting module 1204 from data including A/V informationand/or enhancement data received from the content provider and stored inthe storing module 1201 and process the data including the A/Vinformation and/or the enhancement data using the information about thewatermark.

The A/V processing module 1206 may decode the A/V signal through anaudio decoder and/or a video decoder to process data received from theapplication module 1205.

The displaying module 1207 may display the data processed by the A/Vprocessing module 1206 on a screen of the receiver in an on screendisplay (OSD) manner.

Hereinafter, content information according to one embodiment of thepresent invention will be described.

The content information extracted through the watermark may includeinformation about a content provider. More specifically, the contentinformation may include ID information of a content provider or abroadcaster, ID information of content, information about a time ofcontent, content life cycle information and/or information aboutenhancement data. In an ACR service using fingerprinting according toone embodiment of the present invention, an ACR query result may be usedas metadata for content acquired via an ACR function.

The CP ID information may mean an identifier for identifying a contentprovider or a broadcaster. According to the embodiment of the presentinvention, the CP ID information may serve as a primary identifier. TheCP ID information may include a collection of a serial number foridentifying a producer name and a producer and/or a serial number foridentifying a country, a region and a service property (genre).According to one embodiment of the present invention, a TS-id used foran ATSC program and system information protocol (PSIP) may be used as CPID information.

The content ID information may mean an identifier for identifyingcontent instead of a content provider. More specifically, the content IDinformation may include a collection of a serial number for identifyinga content name and content and a serial number for identifying a contentproperty (genre). According to one embodiment of the present invention,an event-id used for an ATSC PSIP may be used as content ID information.

The CP ID information may be included in the watermark with a frequencyhigher than that of the content ID information and may be configured tobe more easily extracted. In this case, the CP ID information mayinclude more compressed information or more detailed information.

The time information may indicate time information related to content.More specifically, the time information may include a start time, aduration time, an end time and/or a lead time meaning a total serviceplayback time. According to one embodiment of the present invention, thetime information may be based on UTC or GPS and may be configured inanother format such as an STT of an ATSC PSIP. In addition, the timeinformation may be used for synchronization in a display process when aservice such as a synchronized widget is provided. In addition, the timeinformation may be used for content rescheduled by a cable or satellitepersonal video recorder (PVR) set-top box.

The content life cycle information may include information about a lifecycle within which a service is valid.

The information about the enhancement data may include information aboutpresence/absence of enhancement data and usage of a watermark and/orinformation about a path for receiving enhancement data. Morespecifically, the information about the enhancement data may includeinformation about presence of enhancement data provided by a contentprovider and information indicating whether a watermark is used for alicense for preventing illegal copy and distribution, audience ratingsurvey, or content provider recognition for an additional service.Information about a path for receiving enhancement data may includeinformation indicating whether an IP network is used or another mediumsuch as a mobile device is used. More specifically, if enhancement datais received via an IP network, an IP address and/or URL information maybe included and, if enhancement data is received via a mobile device,frequency information, standard (ATSC MH, DMB, media flow, DVB-H, NGH,etc.) information and/or channel information may be included.

FIG. 13 is a diagram showing a watermark extracting process according toan embodiment of the present invention.

A receiver 1320 such as a general digital TV receiver receives an A/Vsignal from a cable or satellite set-top box 1310 mounted in thereceiver 1320 through an HDMI or component connection. Hereinafter,according to the embodiment of the present invention shown in FIG. 13,operation for extracting a watermark and recognizing a CP will bedescribed in stages in the set-top box 1310 and the receiver 1320.

If an input mode of the receiver 1320 is set to an external input mode,the receiver 1320 may search for a CP ID for identifying a contentprovider or a broadcaster. If the set-top box 1310 is turned on and achannel is tuned to, the receiver 1302 may detect a CP ID. The receiver1302 may detect a content ID and time information of content provided bythe detected content provider. If the channel of the service receivedfrom the set-top box 1310 is changed, the receiver 1320 may newly searchfor a CP ID for identifying a content provider according to the changedchannel.

FIG. 14 is a diagram showing profiling of watermark informationaccording to an embodiment of the present invention.

As described with reference to FIGS. 12 and 13, the receiver accordingto one embodiment of the present invention may recognize a CP using awatermark for transmitting CP information included in A/V content. Inthis case, if the amount of information stored in the watermark isincreased, quality of A/V content into which the watermark is insertedmay be deteriorated. Accordingly, in the present invention, in order tosolve such a problem, a method of profiling and leveling a watermarkaccording to attributes of information stored in the watermark andinserting the leveled watermark into A/V content is proposed. In thiscase, since the leveled watermark is differentiated according toattributes of the stored information and is inserted into the A/Vcontent, the amount of information about the inserted watermark may beadjusted to prevent the quality of content from being deteriorated.

Accordingly, in the present invention, in order to level a watermarkaccording to profiling, the watermark may be divided into a basicprofile watermark and an extended profile watermark according toattributes of information stored in the watermark. The basic profilewatermark may include information essentially necessary to recognize areal-time broadcast program in order to perform an ACR function. Theextended profile watermark may include additional information instead ofinformation essentially necessary to perform an ACR function. Aplurality of extended profile watermarks may be present and may beleveled again according to attributes of information. In this case, thebasic profile watermark and the extended profile watermark may beinserted into A/V content according to different cycles.

FIG. 14 is a diagram showing a process of inserting a leveled watermarkinto A/V content as described above.

As described above, a content provider 1400 may receive an originalsignal 1410 such as video/audio, search for a region into which awatermark will be inserted (1401) and embed, in a watermark, metadata1420 related to content to be provided (1402). Thereafter, the contentprovider 1400 may transmit the watermarked signal 1430 to a receiver. Inthe watermarked signal 1430, the original signal 1410 and the metadata1420 may be stored in the form of a watermark. In one embodiment, thewatermark of FIG. 14 may include a basic profile watermark W₁ 1440 andfirst and second extended profile watermarks W₂ and W₃ 1450 and 1460.

The basic profile watermark 1440 may include a content provider IDelement and an extension indicator.

The content provider ID element may indicate information for identifyinga broadcaster or a CP for providing content.

The extension indicator element may indicate whether a watermark of anext level, that is, a first extended profile watermark 1450, isadditionally inserted. If the value of the extension indicator elementis 1, this indicates that the first extended profile watermark 1450 ofthe next level is inserted.

The first extended profile watermark 1450 may include a content IDelement, a timestamp element and an extension indicator element.

The content ID element may indicate serial number information foridentifying a content name or content.

The timestamp element indicates time information of content.

The extension indicator element may indicate whether a watermark of anext level, that is, a second extended profile watermark 1460, isadditionally inserted as described above. As shown in FIG. 14, thesecond extended profile watermark 1460 may include a watermark insertionusage element, a URL element of additional information and an extensionindicator element.

The watermark insertion usage element may indicate information aboutwatermark insertion usage.

The URL element of the additional information may indicate additionalinformation or URL information for receiving a service.

The extension indicator element may indicate whether a watermark of anext level is inserted.

As shown in FIG. 14, a content provider or a broadcaster maydifferentiate the number of times of inserting the watermark into theA/V content according to watermark profiling when the leveled watermarkis inserted into the A/V content. For example, since the basic profilewatermark 1440 includes important basic information of a broadcastprogram, the content provider or the broadcaster may set the number oftimes of inserting the basic profile watermark such that the receiveraccurately recognizes information as soon as possible.

That is, the content provider or the broadcaster may insert basicprofile watermarks 1440 more than extended profile watermarks 1450 and1460 during a transmission cycle of the watermarked signal 1430.Accordingly, the receiver may preferentially extract the basic profilewatermark 1440 and then extract the extended profile watermarks 1450 and1460 as necessary. In this case, the receiver may check presence of theextended profile watermark according to the value of the extensionindicator element and extract the extended profile watermark. That is,the receiver may discriminatorily extract watermarks discriminatorilyinserted according to the profile of the watermark and efficiently storeand manage information included in the watermark.

Up to now, a method and system for providing an ACR service usingwatermarking according to an embodiment of the present invention hasbeen described. Hereinafter, a method of embedding a watermark incontent at a content provider and a method of extracting a watermarkfrom content at a receiver according to an embodiment of the presentinvention will be described in detail.

A method of embedding a watermark in a video frame at a content providerand a method of extracting a watermark from content at a receiver willbe described with reference to FIGS. 15 to 17 and a method of embeddinga watermark in an audio frame at a content provider and a method ofextracting a watermark from content at a receiver will be described withreference to FIG. 18.

FIG. 15 is a diagram showing a method of embedding a watermark in avideo frame according to an embodiment of the present invention.

A content provider first subdivides one video frame 1500 into subframes1501, searches for the subdivided subframes 1502, and determines an areain which a watermark will be appropriately embedded. If a subframe areain which a watermark will be embedded is determined, the watermark isembedded in the subframe area (1510) and the subframes are combined toregenerate one video frame (1520).

In order to extract the watermark from the video frame embedded in theoverall area of one video frame at the receiver, first, the receiversubdivides one video frame into subframes. An area in which thewatermark may be embedded is detected from the subdivided subframes. Thewatermark is extracted from the detected subframe area and the extractedwatermarks are compared to finally determine one watermark. Enhancementdata may be downloaded using content information through the determinedwatermark.

FIG. 16 is a diagram showing a method of embedding a watermark in avideo frame according to another embodiment of the present invention.

More specifically, FIG. 16 shows a method of embedding a watermark in abar data area of a video frame according to an embodiment of the presentinvention.

If a watermark is embedded in the overall area of one video frame,watermarking robustness may be improved but processing necessary toextract the watermark at the receiver may be increased. Therefore, byembedding the watermark in some areas of the video frame, it is possibleto decrease processing necessary to extract the watermark at thereceiver so as to improve operation efficiency of the receiver. Someareas, in which the watermark will be embedded, of the overall area ofthe video frame may be predefined according to the rules of thetransmitter/receiver. Hereinafter, according to the embodiment of thepresent invention, the content provider may embed the watermark in thebar data area which is a part of the overall area of the video frame.

When the content provider transmits content to the receiver, if an imageratio of content mismatches a transmission format, a bar data area 1600may be generated on a screen. For example, as shown in FIG. 16, in caseof converting a video frame having an aspect ratio of 16:9 into anaspect ratio of 4:3, black bands may be generated in upper and loweredges of a screen having the aspect ratio of 4:3 so as not to crop theleft and right edges of the screen. The area of the black band may bethe bar data area 1600 which is called a letterbox.

As shown in FIG. 16, in case of a video frame in which the bar data area1600 is present, the watermark may be embedded in the bar data area 1600to be transmitted. The method of embedding the watermark in the bar dataarea may be performed equally to the process of embedding the watermarkin the video frame described with reference to FIG. 15.

More specifically, the content provider may search for the bar data area1600 of the video frame, in which the watermark will be embedded, (1600)and embed the watermark in the bar data area 1600 (1620).

In order to extract the watermark embedded in some areas of the videoframe, first, the receiver subdivides one video frame into subframes. Anarea in which the watermark may be embedded is detected from thesubdivided subframes. According to the embodiment of the presentinvention shown in FIG. 16, the receiver may detect the bar data areafrom the subdivided subframes. The watermark is extracted from thedetected subframe area and the extracted watermarks are compared tofinally determine one watermark. Enhancement data may be downloadedusing content information through the determined watermark.

FIG. 17 is a diagram showing a method of embedding a watermark in avideo frame according to another embodiment of the present invention.

More specifically, FIG. 17 shows a method of embedding a plurality ofwatermarks including different content information (or metadata) intodifferent areas of one video frame.

For example, as shown in FIG. 17, a watermark including CP IDinformation may be embedded in a logo area 1700 located at a right upperside of a screen and a watermark including a content ID and timestampinformation may be embedded in a central area 1701 of the same screen.

A method of embedding the plurality of watermarks including differentcontent information into one video frame may be largely divided into twomethods according to the embodiment of the present invention.

A plurality of different watermarks may be embedded in different areas1700 and 1701 of a video frame. At this time, the two or more areas 1700and 1701 of the video frame, in which the watermark will be embedded,may be predefined according to the rules of the transmitter/receiver.

Alternatively, a separate watermark including information about thedifferent areas 1700 and 1701 of the video frame may be first embedded.Different areas 1700 and 1701, in which the watermarks including contentinformation will be embedded, may be detected through a separatewatermark and the watermarks may be respectively embedded in the areas1700 and 1701. The separate watermark may be embedded in a specific areaor a logo part of the video frame and may be predefined according to therules of the transmitter/receiver.

If the received video frame includes watermarks having different contentinformation according to areas, first, the receiver acquires informationabout the area, in which the watermark is embedded, through another pathor according to the rules of the transmitter/receiver. Using theacquired information, the watermark is extracted from each area of thevideo frame. According to the embodiment of the present invention shownin FIG. 17, the watermark including information about a content providermay be extracted from the logo area 1700 and the watermark including acontent ID and time information may be extracted from the central area1701 of the screen. Alternatively, if information for extractingdifferent watermarks from the extracted watermarks (e.g., informationabout an area in which the watermark is embedded) is included, it ispossible to extract the different watermarks using the information.

FIG. 18 is a diagram showing a method of embedding a watermark in anaudio frame according to an embodiment of the present invention.

More specifically, FIG. 18 shows a process of embedding one watermark inan audio frame.

An original audio signal is divided into frames having a constant length(1800) and the divided audio frame is subdivided into subframes (1801).Using the subdivided subframes, an area in which a watermark will beembedded may be searched for (1802). For example, a frequency bandhigher than 4 kHz to which a human ear is sensitive may be searched forand determined as a watermark embedding area. The watermark is embeddedin the searched area (1803) and the subframes are recombined into oneaudio signal (1803).

In addition, according to the embodiment of the present invention, aplurality of watermarks may be embedded in an audio signal based on thewatermark embedding process shown in FIG. 18. The embedding method mayinclude a method of respectively embedding watermarks to two or moredifferent frequency bands and recombining the subframes into one audiosignal and a method of embedding one watermark in continuous audioframes and embedding the other watermarks to different frequency bands.

According to the above-described embodiment of the present invention, ifthe audio signal in which the watermark is embedded is received, thereceiver may first divide the audio signal into frames having a constantlength in order to extract the watermark from the audio signal. Thedivided frame may be subdivided into subframes and the watermarkextraction area may be detected from the subframes. The watermark isextracted from the detected area. If a plurality of watermarks isembedded in an audio signal, the watermarks may be extracted from thereceived audio signal at two or more frequency bands defined accordingto the rules of the transmitter/receiver. Alternatively, one watermarkmay be extracted from continuous audio frames and the other watermarksmay be extracted from the signals of different frequency bands definedaccording to the rules of the transmitter/receiver. In addition,according to the embodiment of the present invention of the method ofembedding the watermark described with reference to FIGS. 15 to 18, thewatermark may be embedded in both audio content and video content.

If a watermark including the same content information may be embedded inboth video and audio frames, the same watermark may be repeatedlyembedded in the video and audio frames with a predetermined periodicity.According to such a method, since the receiver may simultaneouslyextract the same watermark from the video and audio frames in which thewatermark is embedded, it is possible to improve accuracy of a result ofextracting the watermark and to reduce a time consumed to finallyacquire an accurate result.

Alternatively, watermarks may be divided according to contentinformation included in the watermark and the divided watermarks may beembedded in video and audio frames. For example, the content informationincluded in the watermark is divided into essential information forrecognizing a broadcast program, such as a CP ID and a content ID, andadditional information. The essential information may be embedded in avideo frame and additional information may be embedded in an audioframe. On the contrary, the essential information may be embedded in anaudio frame and additional information may be embedded in a video frame.Since the amount of information embedded in one video or audio frame isrestricted, if the watermark is embedded using such a method, it ispossible to increase the amount of metadata embedded during apredetermined time.

FIG. 19 is a diagram showing a method of embedding a profiled watermarkaccording to an embodiment of the present invention.

As described with reference to FIG. 14, since the quality of A/V contentmay be deteriorated as the amount of content information stored in thewatermark is increased, if the watermark is leveled according to type ofinformation stored in the watermark, it is possible to efficientlyutilize the watermark while maintaining the quality of content.

According to the embodiment of the present invention, after thewatermark is profiled according to attributes of information stored inthe watermark, the number of times of embedding the watermark in acontent frame during a predetermined cycle may be differently set.

For example, information stored in the watermark may be divided intoessential information for recognizing a broadcast program and additionalinformation or information having a short change cycle and informationhaving a long change cycle. The number of embedding the watermarkincluding essential information and/or information having a short changecycle in the frame during a predetermined cycle may be increased and thenumber of embedding the watermark including additional informationand/or information having a long change cycle in the frame during apredetermined cycle may be decreased.

More specifically, as shown in FIG. 19, a watermark includinginformation necessary to recognize a program such as a CP ID and acontent ID may be embedded in a logo area 1900 and a watermark includingadditional information such as URL information may be embedded inanother specific area 1901 of the frame. In this case, the watermark ofthe logo area 1900 is data necessary to recognize the program and thusmay be embedded in all video frames 1910, 1920 and 1930 and thewatermark of the specific area 1901 of the frame for providing theadditional information may be embedded in the two frames 1910 and 1930once.

FIGS. 20 and 21 show a method of embedding and extracting a watermarkaccording to an embodiment of the present invention.

FIG. 20 is a diagram showing a method of embedding watermarks incontinuous frames according to an embodiment of the present invention.

FIG. 21 is a diagram showing a method of extracting watermarks insertedinto continuous frames according to an embodiment of the presentinvention.

The amount of content information which may be embedded in one A/V frameis restricted. According to the embodiment of the present inventionshown in FIGS. 20 and 21, the watermark may include a large amount ofcontent information.

As shown in FIG. 20, the content provider may embed the watermarkincluding content information into continuous frames. In order toindicate the start and end of the content information, after a start tag2001 indicating start information of content information and an end tag2002 are embedded in content information, the content information isdivided into sub content information 2003. The watermarks including thesub content information 2003 may be embedded in continuous A/V frames200, respectively. In addition, if the watermarks are repeatedlyembedded in the continuous A/V frames 2000 during a predetermined cycle,the receiver may efficiently extract the watermarks.

As shown in FIG. 21, the receiver may extract the watermarks from thecontinuous frames in which the watermarks are embedded. First, thereceiver extracts the watermarks embedded in the continuous A/V frames.A start tag 2101 and an end tag 2102 of content information may bedetected from the extracted watermarks. If the start information of thecontent information is acquired from the start tag 2101, the watermarksare extracted from the A/V frames received after the start tag 2101 hasbeen detected so as to acquire sub content information 2103, and theacquired sub content information 2103 is combined. If the endinformation of the content information is acquired from the end tag2102, the process of combining the sub content information 2103 isfinished to acquire one piece of content information.

FIG. 22 is a diagram showing a method of embedding a watermark in a 3Dvideo frame according to an embodiment of the present invention.

A full 3D video image may be implemented by acquiring a left image and aright image using two cameras, enabling a left eye to view the leftimage and enabling a right eye to view the right image. In the presentinvention, a method of embedding and extracting a watermark in and froma 3D video frame is proposed.

As shown in FIG. 22, the 3D video frame may include a left frame 2200and a right frame 2201. A content provider may embed content information(or metadata) 2202 into the left frame 220 and the right frame 2201 andgenerate a left frame, in which the watermark is embedded, (hereinafter,referred to as a watermarked left frame) 2203 and a right frame, inwhich the watermark is embedded, (hereinafter, referred to as awatermarked right frame) 2204.

A standard definition (SD) or high definition (HD) image may begenerated and transmitted.

First, the content provider may embed the watermark in only the leftframe 2200 or the right frame 2201 and transmit the watermarked left andright frames to the receiver. The receiver may extract the watermarkfrom one of the watermarked left frame 2203 or the watermarked rightframe 2204. In this case, as described with reference to FIGS. 15 to 17,the method of embedding the watermark in the video frame and the methodof extracting the watermark from the video frame may be applied. Thewatermarked video frame may be predefined according to the rules of thetransmitter/receiver.

Second, the content provider may embed the watermark including the samecontent information into the left frame 2200 or the right frame 2201 andtransmit the watermarked left and right frames to the receiver. Thereceiver may extract the watermark from each of the watermarked leftframe 2203 or the watermarked right frame 2204. The extracted watermarksmay be compared and corrected if the watermarks are identical or if adifference between the watermarks is within an allowed error range. Inthis case, the methods described with reference to FIGS. 15 to 17 areapplicable to the method of inserting the watermark into the left frame2200 or the right frame 2201 and the method of extracting the embeddedmark. The watermark may be embedded in the left frame 2200 or the rightframe 2201 using the same method or different methods. Although such amethod may increase complexity when the receiver extracts the watermark,watermark extraction robustness may be increased.

Third, the content provider may embed different watermarks in the leftframe 2200 or the right frame 2201 and transmit the watermarked left andright frames to the receiver. The receiver may extract the watermarkfrom each of the watermarked left frame 2203 or the watermarked rightframe 2204. The extracted watermark may include one piece of contentinformation. Alternatively, content information included in theextracted watermark may be combined to generate one piece of new contentinformation. In the latter case, content information greater in amountthan the amount of content information transmittable via one video framemay be embedded in the video frame. Alternatively, content informationwhich may be included in one video frame may be divided and embedded inthe left frame 2200 or the right frame 2201. In this case, it ispossible to reduce a possibility that the quality of image isdeteriorated.

FIG. 23 is a diagram showing the structure of an ACR service provisionsystem according to an embodiment of the present invention.

More specially, FIG. 23 shows an ACR service provision system forrecognizing broadcast content using both watermarking and fingerprintingtechnologies according to one embodiment of the present invention.

In watermarking, a recognition ratio of content embedded in one frame ishigh but the quality of content may be deteriorated if a large amount ofcontent information is embedded in a watermark. That is, the amount ofembedded metadata is restricted.

In fingerprinting, the quality of content is not influenced but an ACRserver for recognizing content is necessary. In addition, a timeconsumed to recognize content and a content recognition result dependson the size of a reference set.

According to the embodiment of the present invention shown in FIG. 23,if broadcast content is recognized by a combination of watermarking andfingerprinting technologies, it is possible to efficiently provide anACR service while solving the problems of the two technologies.

The present invention includes a method of recognizing a broadcastprogram which is currently being viewed using two technologies.Metadata, which is used to recognize a program such as an identifier ofa content provider or a broadcaster and needs to be most accuratelyrecognized, may be embedded in broadcast A/V content and may betransmitted.

Hereinafter, a mechanism between components of an ACR service provisionsystem shown in FIG. 23 will be described in detail.

If the content provider 2300 transmits watermarked A/V content andenhancement data to an MVPD 2301 (2310, 2330), the receiver 2303 mayreceive the A/V content through a set-top box 2302 but may not receivethe enhancement data (2320). Accordingly, in order to enable thereceiver 2303 to receive the enhancement data through another path, thecontent provider 2300 may transmit additional information of the A/Vcontent to an ACR server 2304 (2340) and transmit the enhancement datato an application server 2305 (s250). The additional information of theA/V content may be information for delivering the enhancement datarelated to the A/V content to the receiver 2303 and may include acontent ID, a timestamp, a URL, etc.

A watermark extractor 2306 of the receiver 2303 may extract a watermarkfrom the received A/V content and, at the same time, a fingerprintextractor 2307 may extract a fingerprint signature from the A/V content.The receiver 2303 may transmit a query including the content informationincluded in the watermark and the fingerprint signature to an ACR server2304 (2360) and acquire the additional information of the A/V contentfrom the ACR server 2304 (2360). Then, the receiver 2303 may receive theenhancement data from the application server 2305 using the additionalinformation of the A/V content (2380). By including the watermarkingresult in the query, it is possible to reduce a time consumed torecognize the content at the ACR server and to improve contentrecognition accuracy of the ACR server.

FIG. 24 is a diagram showing a method of providing an ACR service usinga broadcaster a logo according to an embodiment of the presentinvention.

FIG. 25 is a diagram showing a broadcaster logo according to anembodiment of the present invention.

FIG. 26 is a diagram showing an ACR service using a broadcaster logoaccording to an embodiment of the present invention.

A content provider (or a broadcaster) has a logo indicating contentthereof. For example, content providers may have logos shown in FIG. 25.The broadcaster or the content provider may insert a logo into a rightupper side, a left upper side, a right lower side or a left lower sideof a screen as shown in FIG. 26 so as to display content. Even when avideo scene displayed on the screen is changed, the logo is not changedand is continuously displayed. In the present invention, an ACR servicemethod of automatically recognizing information continuously displayedin a specific part of a screen using the property of a logo is proposed.

As shown in FIG. 24, according to the embodiment of the presentinvention, a watermark may be inserted into a logo to provide an ACRservice. A receiver may recognize the logo to acquire content providerinformation indicating the logo and access the content provider toreceive a service, thereby providing an ACR service. Hereinafter, amethod of providing an ACR service using a logo will be described indetail.

First, the receiver may acquire logo information through a logorecognition step 2401 and a logo comparison step 2402. The receiver mayacquire content provider information as a comparison result to recognizethe content provider (2403) and access the content provider server(2404). Additional information of the content may be received from thecontent provider server (2405) to provide an additional server (2406).

The logo recognition step 2401 and the logo comparison step 2402 of thereceiver may be performed by comparing the logo recognized by thereceiver and the logos of the content providers pre-stored in thereceiver.

Information about the stored logos is information received and stored bythe receiver through an IP network or MH or default information storedin the receiver. New version information of the logo may be maintained.In case of information received via the IP network or MH, new logoinformation may be maintained by continuously receiving logo informationthrough the IP network or MH and, in case of the default information,new logo information may be maintained through continuous update.

In the logo comparison step 2402, if the recognized logo is comparedwith all logo information stored in the receiver, the amount ofinformation is large. Thus, a burden is imposed on the receiver.According to the embodiment of the present invention, since extraction,storage and comparison may be performed based on the detailed propertiesof the logo, it is possible to overcome such a restriction. For example,logo properties such as logo color, letter (SBS, CNN, etc.) and contourmay be recognized to perform comparison. If a contour is used, a spacein which the logo is displayed may be divided by coordinates torecognize the logo using the coordinate properties.

FIG. 27 is a diagram showing an ACR service provision method using alogo and a watermark according to an embodiment of the presentinvention.

As shown in FIG. 27, in an ACR service, a receiver may extract logoinformation 2701 or a watermark 2702 to acquire information about acontent provider and receive an additional service provided by a contentprovider through an IP network or MH.

As described above, if a watermark is embedded in content, the data ofthe content may be changed. Accordingly, in the present invention, amethod of embedding a watermark in a logo area included in content isproposed.

Hereinafter, a method of utilizing logo information and a watermark at areceiver for an ACR service according to the embodiment of the presentinvention shown in FIG. 25 will be described in detail.

As described above, the receiver according to the embodiment of thepresent invention may extract and utilize a watermark including apredetermined amount of content information by a specific algorithminserted into a specific field of A/V frame data when content is playedback. For example, the receiver which receives video content including30 frames per second may extract, from M frames, a watermark includingdata of N bits corresponding to a specific field or a specific locationin each frame. In this case, the total amount of extracted watermarkdata may become N*M bits. If the number of watermarked frames isincreased, the amount of watermark data to be extracted is increased.Therefore, a time consumed to extract the watermark at the receiver maybe increased and the quality of the A/V content processed by thereceiver may be deteriorated.

Accordingly, in the present invention, a method of embedding a watermarkin a logo area included in a video frame is proposed. The logo area isincluded in all video frames and may be displayed at the same locationon a screen even when a playback screen is changed. Accordingly, evenwhen the amount of data is large, if a watermark is embedded in a logoarea, it is possible to reduce a time consumed to extract a watermarkand to decrease a possibility that the quality of A/V content isdeteriorated.

Hereinafter, a method of embedding a watermark in a logo according toone embodiment of the present invention will be described. The method ofembedding the watermark in the logo according to the embodiment of thepresent invention may include a method of embedding bar code data at apredetermined location of a logo, a method of utilizing a metadatapacket of logo information as a watermark and a method of embedding awatermark in a part of image data of a logo.

In case of the method of embedding the bar code data at thepredetermined location of the logo, the receiver may recognize the barcode data located at a specific location as a watermark. In this case,the bar code data is not displayed.

In the method of utilizing the metadata packet of the logo informationas the watermark, the metadata packet describing the logo is used as thewatermark while the original form of image data of the logo ismaintained. In this case, the metadata packet may include a CP ID, acontent ID, time information, etc.

In the method of embedding the watermark in the logo information, N logoframes are combined using bits of the part of the image data of the logoand N watermark bits are embedded in the logo image data. In this case,content information or metadata included in the watermark is asdescribed above.

FIG. 28 is a diagram showing a method of processing an ACR servicerelated to a broadcast program according to an embodiment of the presentinvention.

More specifically, FIG. 28 is a flowchart illustrating operation of acontent provider according to the embodiment of the method of processingthe ACR service related to the broadcast program of the presentinvention described with reference to FIGS. 15 to 22.

The content provider may generate a watermark (S2801)

The watermark may include channel information related to content and/orcontent information having URL information of content additionalinformation. More specifically, as described with reference to FIG. 12,the watermark may include ID information of the content provider or thebroadcaster and/or the ID information of the content. In addition, asdescribed with reference to FIG. 14, the watermark may be leveledaccording to attributes of information included in the watermark.

The content provider may embed the generated watermark in the content(S2802).

The content provider may embed the watermark in a video frame and/or anaudio frame.

If the watermark is embedded in a video frame, as described withreference to FIG. 15, the content provider may subdivide the frame ofthe video content into subframes and embed the watermark in thesubdivided subframes. As described with reference to FG. 16, the contentprovider may embed the watermark in a certain area of the video frame.As described with reference to FIG. 17, the content provider may embed aplurality of watermarks including different content information intodifferent areas of one video frame.

If the watermark is embedded in an audio frame, as described withreference to FIG. 18, the content provider may subdivide one audio frameinto subframes and embed the watermark in the subdivided subframe. Asdescribed with reference to FIG. 18, the content provider may embed aplurality of watermarks in one audio signal.

As described with reference to FIGS. 15 to 18, the content provider mayembed the watermark in both audio content and video content. Asdescribed with reference to FIG. 20, the content provider may embed thewatermark in continuous frames. As described with reference to FIG. 22,the content provider may embed the watermark in 3D content. As describedwith reference to FIGS. 24 to 27, the content provider may embed thewatermark in the logo area.

If the watermark is leveled according to attributes of contentinformation, as described with reference to FIG. 19, the contentprovider may change the cycle according to content information and embedthe watermark in the frame. In this case, as described with reference toFIGS. 14 and 19, the leveled watermark may be embedded in the A/Vcontent by changing the number of times of embedding the watermarkaccording to level.

The content provider may transmit the content including the watermark(S2803).

FIG. 29 is a diagram showing a method of processing an ACR servicerelated to a broadcast program according to another embodiment of thepresent invention.

More specifically, FIG. 29 is a flowchart illustrating operation of areceiver according to the embodiment of the method of processing the ACRservice related to the broadcast program of the present inventiondescribed with reference to FIGS. 15 to 22.

The receiver may receive content including a watermark (S2901).

The watermark may include channel information related to content and/orcontent information having URL information of content additionalinformation. More specifically, as described with reference to FIG. 12,the watermark may include ID information of the content provider or thebroadcaster and/or the ID information of the content. In addition, asdescribed with reference to FIG. 14, the watermark may be leveledaccording to attributes of information included in the watermark.

The receiver may extract the watermark from a video and/or audio frame(S2902).

As described with reference to FIG. 28, the content provider may embedthe watermark in the video and/audio frame. The receiver may receive thevideo and/or audio frame, in which the watermark is embedded, andextract the watermark. More specifically, as described with reference toFIG. 5, the receiver may use the micro-code to extract the watermark.

If the watermark is embedded in a video frame, as described withreference to FIG. 15, the receiver may subdivide one video frame intosubframes, detect the subframe in which the watermark is embedded andextract the watermark. As described with reference to FIG. 16, thereceiver may detect the bar data area from the subdivided subframe andextract the watermark. As described with reference to FIG. 17, thereceiver may acquire information about the area, in which the watermarkis embedded, and extract the watermark from each area of the video frameusing the acquired information, if the video frame includes thewatermark including content information which differs according toareas.

If the watermark is embedded in an audio frame, as described withreference to FIG. 18, the receiver may subdivide the audio signal intosubframes, detect the watermark extraction area and extract thewatermark.

As described with reference to FIG. 21, if the watermark is extractedfrom the continuous frames, the receiver may extract sub contentinformation from the continuous frames, combine the sub contentinformation using the start tag and the end tag, and acquire one pieceof content information. In addition, as described with reference to FIG.22, if the watermark is embedded in the 3D content, the receiver mayextract the watermark from the left and/or right frame of the 3Dcontent.

If the watermark is leveled according to attributes of contentinformation, as described with reference to FIG. 19, the contentprovider may change the cycle according to content information and embedthe watermark in the frame. In this case, as described with reference toFIGS. 14 and 19, the receiver may discriminatorily extract thewatermarks discriminatorily embedded according to the profile of thewatermark.

The receiver may acquire additional information through an IP networkusing the extracted watermark (S2903).

More specifically, as described with reference to FIG. 23, the method ofprocessing the ACR service related to the broadcast program of thepresent invention may include the method of processing the ACR serviceby combining watermarking and fingerprinting.

According to the present invention, it is possible to identifyinformation regarding a broadcast program in real time using ACRtechnology.

According to the present invention, it is possible to receiveenhancement data related to a broadcast program using an ACR function soas to have an improved TV viewing experience.

According to the present invention, by recognizing information regardinga broadcast program and receiving enhancement data using watermarkingtechnology, a TV receiver can perform an ACR function alone without aseparate ACR server.

According to the present invention, it is possible to profile awatermark according to information included in the watermark andefficiently utilize watermarking even when the amount of informationincluded in the watermark is large.

According to the present invention, by combining watermarking andfingerprinting, it is possible to efficiently provide an ACR service.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present inventionwithout departing from the spirit or scope of the inventions. Thus, itis intended that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents.

1-20. (canceled)
 21. A method of providing an interactive servicerelated to a content, the method comprising: receiving the contenthaving plural watermarks, wherein the watermarks are inserted in videoframes of the content, wherein at least one of the frames include afirst-level watermark, wherein at least one of the frames including thefirst-level watermark further includes a second-level watermark as anextension for the first-level watermark, wherein the first-levelwatermark includes major information for the interactive service of thecontent, and a first extension indicator, wherein the first extensionindicator indicates whether the first-level watermark has extension,wherein the second-level watermark includes minor information for theinteractive service of the content; extracting the first-level watermarkto obtain the major information for the interactive service, and thefirst extension indicator; deciding whether the minor information areneeded for providing interactive service; extracting the second-levelwatermark based on the first extension indicator to obtain the minorinformation for the interactive service; providing the interactiveservice of the content by using the extracted information.
 22. Themethod of claim 21, wherein a period between frames having first levelwatermarks are different from a period between frames having bothfirst-level watermarks and second-level watermarks.
 23. The method ofclaim 21, wherein the first-level watermark includes an identificationinformation for source of the content.
 24. The method of claim 21,wherein the second-level watermark includes a timestamp, and wherein thetimestamp establishes a time base for synchronizing the interactiveservice with the content.
 25. The method of claim 21, wherein thefirst-level watermark is inserted in a logo image area of the videoframe, and wherein the second-level watermark is inserted in a symbolimage area of the video frame.
 26. The method of claim 23, wherein atleast one of the frames including the second-level watermark furtherincludes a third-level watermark as an extension for the second-levelwatermark, and wherein the second-level watermark further includes asecond extension indicator indicating whether the second-level watermarkhas extension.
 27. The method of claim 26, wherein the third-levelwatermark includes an URL information for additional data of theinteractive service.
 28. The method of claim 27, wherein the methodfurther includes, deciding whether the URL information for additionaldata are needed for providing interactive service; extracting thethird-level watermark based on the second extension indicator to obtainthe URL information for additional data; obtaining the additional dataover an IP (Internet Protocol) network by using the extracted URLinformation; and providing the interactive service of the content byusing the obtained additional data.
 29. A method of providing aninteractive service related to a content, the method comprising:generating plural watermarks; inserting the generated watermarks intovideo frames of the content, wherein at least one of the frames includea first-level watermark, wherein at least one of the frames includingthe first-level watermark further includes a second-level watermark asan extension for the first-level watermark, wherein the first-levelwatermark includes major information for the interactive service of thecontent, and a first extension indicator, wherein the first extensionindicator indicates whether the first-level watermark has extension,wherein the second-level watermark includes minor information for theinteractive service of the content, wherein the major information andthe minor information for the interactive service are used to providethe interactive service of the content; and transmitting the contenthaving the watermarks.
 30. The method of claim 29, wherein a periodbetween frames having first level watermarks are different from a periodbetween frames having both first-level watermarks and second-levelwatermarks.
 31. The method of claim 29, wherein the first-levelwatermark includes an identification information for source of thecontent.
 32. The method of claim 29, wherein the second-level watermarkincludes a timestamp, and wherein the timestamp establishes a time basefor synchronizing the interactive service with the content.
 33. Themethod of claim 29, wherein the first-level watermark is inserted in alogo image area of the video frame, and wherein the second-levelwatermark is inserted in a symbol image area of the video frame.
 34. Themethod of claim 31, wherein at least one of the frames including thesecond-level watermark further includes a third-level watermark as anextension for the second-level watermark, and wherein the second-levelwatermark further includes a second extension indicator indicatingwhether the second-level watermark has extension.
 35. The method ofclaim 34, wherein the third-level watermark includes an URL informationfor additional data of the interactive service.
 36. The method of claim35, wherein the URL information for additional data is used forobtaining the additional data over an IP (Internet Protocol) network,and wherein the additional data is used for providing the interactiveservice of the content.